This PR implements a streaming chat API endpoint using FastAPI and Ollama LLM, featuring real-time response streaming with JSON formatting and completion signaling.

Changes

• Added streaming chat endpoint with FastAPI
• Implemented Ollama client integration
• Added JSON response formatting with completion status
• Added Ollama server setup script

New Files

• main.py: FastAPI server implementation with streaming endpoint
• ollama_client.py: Ollama integration and chat handling
• ollama.sh: Server setup and initialization script

Before hitting endpoint

pull ollama
./ollama.sh pull

start ollama
./ollama.sh start

for more
./ollama.sh --help

API Endpoint

• POST /chat
• Request Body:

  {
    "message": "string"
  }
  
  
  #TODO: vector embeddings for input query
  refer `/llm/OllamaService.py:ollama_client method` left an comment for more help